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FOREWORD 



You may have noticed editorial and format improvements in the last 
edition of this publication. These have occurred through closer communi- 
cation and sharing of skills among the Department of Education and those 
six states* involved with the West Virginia RTC, We hope that with future 
publication of "Observations" the quality of output will be further en- 
hanced. 

Thus far there has been little feedback from you, the reader. We 
wonder whether these articles have been helpful to you and would appreci- 
ate your comments and criticisms. We hope you will share your views with 
the West Virginia RTC staff by forwarding them to Richard Nida. 

Our mailing list is new and, therefore, undergoing constant revision 
and updating. We suspect there are many in our intended readership who 
are not on our list. If you are aware of such omissions we ask that you 
send the new names and addresses to the West Virginia RTC. If the publi- 
cation is being sent to someone no longer in your office, please forward 
it to them. 

Finally, we hope you will share your copy with your associates so 
that they can avoid the pitfalls and enjoy the satisfactions of setting a 
program evaluation project on a successful course. 



James E. Taylor, Ph.D. 
Project Officer, RSA 
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PREFACE 

The following set of manuscripts documents in part the experience of 
six Model Evaluation Units (MEUs) in state agencies for vocational re- 
habilitation. The three-year project began in October of 1978 when MEUs 
were established in the following agencies: 

1. Delaware Division of Vocational Rehabilitation 

2. Michigan Bureau of Rehabilitation 

3. Mississippi Vocational Rehabilitation for the Blind 

4. Oregon Department of Human Resources 

5. Pennsylvania Bureau of Vocational Rehabilitation 

6. Virginia Department of Rehabilitative Services 

The MEUs were funded by the Rehabilitation Services Administration 
and have been assisted in their developmental efforts by the West Virginia * 
Rehabilitation Research and Training Center as coordinating contractor for 
the project. 

The manuscripts in this edition focus upon the expanded capacity for 
evaluation that the MEUs have developed. The Delaware agency's manuscript, 
"Data System Operation," describes the progress of that agency in the area 
of computerization. In its manuscript, "Developing Capacity for Data 
Analysis for Program Evaluation," the Michigan MEU details its experience 
in acquiring appropriate data processing resources. Another manuscript, 
"Computer Acquisition and Operation," describes yet another experience in 
trying to solve data processing problems that of the Mississippi MEU. 
The Oregon agency's manuscript, "Word Processing Applications of the Oregon 
Model Evaluation Unit," discusses the many uses to which the agency has put 
its new word processing system. In "Evaluation Section Library," the 
Pennsylvania MEU explains in detail how it went about developing a tech- 
nical assistance center. And the Virginia MEU in "Planning for an 
Independent Living Program in Virginia" describes how it is planning to 
evaluate the present system for delivering comprehensive rehabilitation 
services for independent living and to develop recommendations for expan- 
sion and Improvement. Finally, the West Virginia Research and Training 
Center has also contributed an article, "A Systematic Approach to Training 
Needs Assessment for Vocational Rehabilitation Program Evaluators," that 
explains how a staff training needs assessment can be completed for a 
program evaluation unit. 



Richard S. Nida, Ph.D, 
Project Officer 
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DATA SYSTEM OPERATION 

Vincent Finelli, Systems Analyst 
Delaware Division of Vocational Rehabilitation 



The Model Evaluation Contract has made it possible for Delaware 
Division of Vocational Rehabilitation (DVR) to make great strides in the 
area of computerization. Prior to the Model, DVR's daily updating turn- 
around time averaged approximately 15 days (see Flowchart) . Compared to 
some batch systems this might seem an acceptable turnaround time, but to 
DVR with its Management by Objectives (MBO) approach, faster turnaround 
time was required. While operating within the framework of the Model 
Contract, dollars were not unlimited, and were therefore a constraint as 
to how equipment would be purchased and utilized to speed up turnaround 
time. As seen in the flowchart, the field input preparation for the daily 
update consumed three days for management signatures, approvals, adminls- 
trivia, etc. .Mailing from the field via the State-Operated Pony Express 
required, on the average, another three days. Coding field input into a 
format ready for keypunching required one full day. Mailing from Head- 
quarters to the Department of Labor's IBM computer (ten miles away) for 
processing consumed one-half day. Actual computer keypunching, updating, 
printing, and bursting output utilized three days and another half -day was 
required for the mail trip to return the computer processed updates to DVR 
Error checking and distribution preparation by DVR's Input/Output Clerk 
required one full day. An finally, the mailing from Headquarters back to 
the field required three days. One complete cycle of the daily update 
required 15 working days. 

Within the constraints of the Model Contract, where was the best place 
to target efforts to improve turnaround time without disrupting field 
staff and working with Delaware's policy against proliferation of computer 
equipment outside of Central Data Processing? 

The most cost-effective place DVR found to improve daily turnaround 
time was the area of day number 7 through day number 12, inclusive* DVR's 
analysis showed that with an on-line capability, the bock marked "Current 
EDP Turnaround Time" (six days) could be reduced to one day which is shown 
in the block marked "New EDP Turnaround Time." It was expected that sav- 
ings would come from eliminating coding, mailing to and from the computer, 
and that processing and checking error lists would be greatly reduced. 

Aside from the on-line information system of the Delaware State Police, 
Delaware State Government had no capability for supporting on-line, real 
time operations. DVR therefore turned to the University of Dealware's 
Burroughs B-7700 time-sharing system. This system was perfectly at home 
with the hybrid on-line, real time remote batch system that DVR was seeking. 
Cost justification was in DVR's favor as analysis showed that the agency 
could maintain data processing at approximately 50 percent of what it was 
presently paying and have on-line, real time capability as well. As a 
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sidelight, along with on-line came SPSS, Reporter II, etc. These were 
added benefits of the Burroughs system that DVR could capitalize on once 
a conversion from IBM to the Burroughs was made. In fact, that conversion 
was made readily and the new system is running quite smoothly. 

DVR's Input/Output (I/O) operation staff had to learn new ways of 
data processing. All of the hand transcribing from one document to another 
was now obsolete. Eliminating the six days at the Department of Labor 
(DOL) meant no more "slack" time and no more putting off report distribu- 
tion and keying until tomorrow as backlogs would result. The jobs of key- 
ing, printing, and bursting of reports which were formerly the jobs of 
the DOL's Computer site were now the responsibility of the I/O operator at 
DVR. Time budgeting throughout the training process was critical during 
the beginning phases of the Burroughs implementation. The new I/O skills 
involved the use of printing terminal, display terminal, dial-up acoustic 
coupler, CANDE (Burroughs editing language), creating computer files, 
merging files, editing files, titling files, signing on and off, etc. 
This is very complex for someone who has had no computer experience. 

After the few initial bugs of conversion were ironed out and training 
sessions of field input and I/O staff were conducted, the statistical com- 
puter system ran smoothly. 

The systems operation looks much like the "old" statistical computer 
system except currently DVR now has three computer terminals and associ- 
ated acoustic couplers. Two of the terminals are 120-characters-per-second 
DEC LA-120's and the third terminal is the Hewlett-Packard video display 
terminal HP 2621A. Additionally, DVR has a CPT-8000 word processor which 
may be used as a backup terminal for the video display or printing terminal. 

The I/O operator receives, batches, keys, edits, runs updates, main- 
tains files, maintains log book, checks edit reports, prints output , and 
answers field questions for the statistical computer system as compared 
to the "old" system where batching, coding, mailing, edit checking and 
distributing were the primary functions. 

In Delaware, the on-line, real time has enabled DVR to drastically 
cut EDP turnaround time from six days to approximately one to two days, 
reduce data processing costs, and provide more accurate and timely in- 
formation to the user. 

The on-line system also provides capability for special studies such 
as those which are currently being conducted in COBOL as well as SPSS to 
support Management's decision making. 

Perhaps the most significant addition DVR has made to its EDP system 
is an on-line, interactive Inquiry program which allows a user to 
efficiently display on a CRT a client's computer record by simply enter- 
ing the Social Security number. 



DEVELOPING CAPACITY FOR DATA ANALYSIS FOR PROGRAM EVALUATION 

MICHIGAN ' S EXPERIENCE 

James Nuttall, M.A. , Program Evaluator 
Michigan Bureau of Rehabilitation 

Rehabilitation agencies vary widely in their capacity to process data. 
Some are still heavily dependent on manual processes while others have 
acquired very sophisticated electronic equipment. 

Michigan's Bureau of Rehabilitation (MBR), like many other state 
agencies, has a large central data processor which generates routine re- 
ports. However, there has been little flexibility to perform technical 
analysis of the type desired by the Program Evaluation Unit, Therefore, 
one of the objectives of the Michigan Model Evaluation Unit was to acquire 
appropriate data processing resources. The report which follows describes 
our efforts and is presented as an illustration which may be of interest to 
other evaluation units with a similar problem. 

MBR: DATA ANALYSIS NEEDS 

As part of our ongoing activities, the Program Evaluation Unit is 
responsible for a number of reports which require special data analysis. 
These reports can be divided into two types; I.e., the routine and the 
specialized. Our routine reports generally are quarterly or annual. 
These reports summarize client characteristics from the R-300 data, client 
follow-up information from the follow-up report, and counselor productivity 
reports. Specialized reports, however, occur as a response to a specific 
request by agency management or as part of our investigative research 
effort. 

The annual reports are generally of a large, if not massive nature. 
They require the summarization of the year's activity of clients or of 
counselors. Frequently these reports are presented for input into the 
legislative process, for public and Bureau Information, or as a Federal 
requirement. Thus, the data are usually presented in a straightforward 
and simplified manner. This often requires summary statistics in the 
form of frequencies, percentages, and graphic displays. Generally these 
reports take a good deal of time to prepare in spite of simplified formats 
which are often standardized. 

The specialized reports, on the other hand, do not have a standard- 
ized format. They involve data of a more specific nature. The data 
generally require more preparation and a greater variety of statistical 
reduction. At times these reports may involve the same data bases as the 
annual reports; i.e., R-300 data or client follow-up information. Examples 
of this type of reports are an analysis of client demographics by geograph- 
ic area, severity of handicap, or income at placement. 



But with increasing frequency, specialized reports are requiring the 
collection and analysis of data tailored to meet the specific issues 
involved. For example, such tasks have involved the development of case 
goals for MBR counselors and the development of client surveys. In these 
cases the ready-made data bases collected by the agency's Management 
Information System do not answer the questions raised by specialized 
requests. 

DATA ANALYSIS AND THE DEPARTMENTAL COMPUTER 

For our routine reports, our data analysis is often done by the State 
of Michigan's Department of Education computer. The department computer 
and its management are typical of many of the data processing centers in 
the state. There are no general statistical or data analytic packages 
available on the system. 

All programs placed on this system are written to generate a specific 
routine report. Any alteration in the format of the report means rewriting 
sections of the program. Since the computer center is over-loaded and 
understaffed, any program revision becomes a "major" reprog ramming effort. 
Therefore, one is reluctant to request reprogramming unless the structure 
of a report is to be altered for an extended period of time. 

Most of the data analytic reports generated by the computer center 
come from the agency's Management Information System. The data base of 
the MIS contains items such as R-300 items, client costs, SSI costs, 
SSTF costs, counselor assignments, and MBR budget information. 

Routine reports presently generated by the computer give us a 
frequency breakdown of all R-300 items for the fiscal year, a listing of 
the number of severe and non-severe clients by diagnostic group, and the 
calculation of average time in statuses and average costs of services for 
closure types. 

As previously mentioned, these reports summarize data by giving the 
frequency and percents associated with data items and the calculation of 
averages in some cases. Such reports have served their purpose well in 
tracking the basic statistics needed for reporting requirements and the 
publication of the Bureau's activities. They have also aided in establish- 
ing a baseline for many data elements. 

However, with the growing press towards higher evaluation standards 
and the management of shrinking resources, our Program Evaluation Unit is 
asked to supply information and analysis which go beyong the scope of the 
routine data analysis. Management questions presented to the unit are 
more frequently of a relational, causal, or predictive nature. 

Such questions cannot be answered by the examination of frequencies 
and percentages. Data must be summarized in a contingency or relational 
manner. Some contingency tables can be generated by the department 



computer but these do not allow for the computation of chi squares, co 
lations, or summary statistics based upon a correlational analysis; i. 
regression, factor analysis, or significant tests between groups. 

Examples of predictive questions presently being asked are as 
follows : 

- What would counselor productivity be (as measured by 26, 28, 
and 30 closures) if the client caseload were all severely dis- 
abled? (Statistical Modeling) 

- Are there any predictive variables in the present data base 
which will tell us at intake how much a given case will 
cost? (Predictive Model) 

- What is the relationship of time in service statuses and the 
clients' reported effectiveness of our service? (Causal Model) 

EVALUATION PROGRAM NEEDS 

The growth of program evaluation has pushed us towards expanding 
data processing capability. What are the basic requirements for meeting 
our expanded evaluation capacity? The answer for meeting the requirem&ri 
of data analysis closely fits the data processing cycle itself. This 
is data input (preparation), data retrieval (storage), data analysis 
(statistical computation) > and timely output (turn-around) as well as cost 
effectiveness. To handle these needs we began to search for additional, 
computer capacity to meet our unit's needs. 

In searching for a data analytic system we concluded that we want&d 
flexible input alternatives. The very small surveys done by our group 
could be placed on a computer by an on-line terminal. Larger projects 
might require key-punching on card by the departmental computer center. 
Then there was the availability of data from our department's computer 
by tape input. Data could possibly be in disk, card, or tape form. 

We had two requirements for data retrieval and manipulation. These 
were the ability to store and to correct data files. This required at tire 
least tape retrieval, but better yet would be disk storage of data sets 
during the correction process. Disk storage generally allows for the 
rapid search of a file and for line-by-line correction. This type of 
manipulation is less cumbersome than tape searching and editing. Howeve*r 
we did require tape storage and retrieval for data sets not frequently 
used or data sets with large numbers of clients, such as a year's total, of: 
R-300 files. 

In looking for data analysis packages, we wanted a system with a 
large number of statistical routines that could handle any of our vario 
data sets both the routine or special. Thus, we wanted a system which. 
had any of the general analytical packages such as Statistical Package 



the Social Sciences (SPSS), Bio-medical Data Package (BMDP) , or Statistical 
Analysis System (SAS) . 

All of these standard packages provide for basic descriptive statis- 
tics such as means, standard deviations, frequencies, variable transforma- 
tions, and correlations. Beyond these basics are non-parametic statistics 
such as chi square, and multivariate statistics such as multiple regression, 
factor analysis, reliability, analysis of variance, and multivariate ana- 
lysis of variance. 

Additionally, we were looking for a data analytic system which would 
allow us to input our programs via an on-line terminal. The advantage of 
this type of system comes in terms of immediacy of feed-back on whether a 
program has run or has "bombed out" with errors. Such error detection 
allows for immediate correction and resubmission of programs. 

Once the program was run, we wanted rapid turn-around time in obtain- 
ing the output. Very often the next step in an analysis depends upon the 
conclusions drawn from the previous analytic step. None of these data 
analytic capabilities was available to use through our state computer 
system. 

OUR SOLUTION 

Fortunately, a solution presented itself through the use of the Univer- 
sity of Michigan's computer facilities. Feasibility discussions were held 
with the Rehabilitation Research Institute at the university, which has 
analyzed the national data tapes of the R-300 for RSA. Since the Institute 
was already familiar with and using the university computer, a training 
program and trial utilization period with the computer was established 
through them. 

Our training program consisted on ten one-day sessions spaced about 
two weeks apart. Spacing the training sessions was very useful since it 
allowed time for reading the numerous and voluminous computer manuals 
and allowed members of the staff to practice the elements of our lessons. 

The sessions began with a basic introduction on how to work with the 
computer via a computer terminal. To start with, we learned how to sign 
on to the computer and to build either a data set or program files. Such 
files are maintained on the large disk system utilized by the computer 
center. 

Our next step was to learn how to correct errors within a data set or 
a program. This is accomplished by using the routine on the computer 
called "editor." The editor allows the terminal operator to enter a 
specified line of a file and change any character or number which is in 
error. The editor has the added advantage of being able to search an 
entire program or data set and alter a character or numeric string which 
is placed throughout the file. This is a great advantage if one wishes 



to change a data element such as an R-300' missing data code of "-H-" to, 
say, "-9." If three hundred of the plus-pluses (-H-) occur, one command 
can change them all to a more acceptable code (-9). 

After we learned to manage our own files, we then began to explore 
the statistical packages available on the computer. These were SPSS, 
OSIRIS, and MIDAS. SPSS, as stated earlier, stands for the Statistical 
Package for Social Science which is produced by Northwestern University. 
This series of statistical routines can be found on a large number of 
university computers and at the West Virginia Rehabilitation Research and 
Training Institute. 

The other computer packages are developed and maintained at the 
University of Michigan. OSIRIS stands for Organized Set of Integrated 
Routines for Investigation in Statistics produced by the University of 
Michigan's Institute for Social Research. The other package, MIDAS, 
stands for for the Michigan Interactive Data Analysis System produced by 
the university's statistical research laboratory. 

All three of these systems have their special advantages. For 
example, the correlation matrix generated by an SPSS run gives the prob- 
ability level below each correlation in the matrix. A factor analysis 
from OSIRIS calculates and prints the percentage of variance contributed 
by each factor. MIDAS offers a cluster program which easily clusters 
variables or clusters subjects, depending upon your interests. 

As can be seen by these examples, as the researcher becomes familiar 
with a number of statistical packages, he/she may choose the specific one 
which best suits the data and the analysis to be performed. It might be 
added that coming by this experience can only be gained by using a number 
o packages, having funds which allow for exploration of your data, and, 
above all, a willingness to learn and experiment with a number of systems. 
The author is fortunate in this respect since he has been able to utilize 
three different university computer systems. The best advice is, if at all 
possible, not to restrict yourself to one analysis package or to one com- 
puter if it doesn't allow multiple analysis alternatives. 

THE SPECIFICS; DATA ENTRY 

In order to work on the university computer which is seventy miles 
from us, we leased a hard copy computer terminal and telephone data trans- 
mission modum. This type of computer terminal prints everything you type, 
as well as all computer responses on paper in the same manner as any type- 
writer. We chose this terminal over a cathode-ray (T.V.) display since 
the printed copy allows us to view our data sets, programs, and small out- 
puts away from the computer. 

With the aid of our graduate assistant and secretary, we planned to 
enter most of the data from our specialized studies ourselves. After 
some experience, we found that a data set with four hundred subjects and 



two data cards per subject generally takes about fourteen hours to enter 
at a computer cost of approximately fifteen dollars. 

The data entry time and cost do not sound very taxing until you try 
to enter and work on a number of data sets say seven or ten. Soon data 
entry becomes a very laborious task, taking up not only terminal time but 
the valuable time of the staff. The additional staff cost usually runs 
between seven to ten dollars per hour. At the average of fourteen hours 
for data entry, the cost in staff time can be as high as one hundred to 
one hundred forty dollars for each set entered. 

At these rates, an alternative for data entry was needed. This meant 
looking for a source which could prepare our data for us. Frequently uni- 
versities, private data processing agencies, and departments within a 
state maintain keypunching services. Our computer service with the 
Department of Education was willing to keypunch data on to cards for us. 
So, in the future we will be utilizing this service as much as possible. 
These cards will be read into the computer using a link between the 
University of Michigan computer and a local computer at Michigan State 
University. 

We have also asked our department computer center to prepare data 
tapes for us. For example, in our recent client follow-up survey, the 
computer center made a copy of the R-300 data on each client who was in 
our sample survey. Thus, we had a tape record of the R-300 files of the 
690 individuals receiving the survey. The tape generated at our depart- 
ment was carried down to the university and placed on its system. 

Having asked for a data tape like those submitted to USA, we encounter- 
ed several difficulties in working with it. The tape did not carry a 
standard IBM label but carried a volume number of six blanks at the 
beginning. 

A tape label put at the beginning of a tape by a computer carries 
instructions as to how a tape is to be read by the computer. This label 
also contains a password which puts a lock on the tape to prevent un- 
authorized use. After much computer footwork, we circumvented the tape 
lock and label. After reading the data into a file, we also discovered 
that the RSA tape format does not carry an end-of-tape mark. However, 
after running several sets of programs which produced some very strange 
results, we found that the end of the R-300 data set is designated by 
having the last six eight-column rows filled with nines. 

EDITING DATA 

After building the data into a file it must be edited for errors or 
for unacceptable data codes. Although the R-300 data of our agency is any- 
where from 95- to 97-percent error-free, the three or five percent of 
errors can cause a program to fail to run. One can simply delete cases 
which have erroneous codes. However, a five-percent error rate in our 



follow-up sample of 690 cases with R-300 data would mean the loss of 
thirty-five cases. This might be tolerable, but one dislikes losing the 
entire data on a subject because of a single error in the data. 

Generally, we have found that ten percent of our cases have a wild 
code which needs correcting. The computer routine of the editor is able 
to scan and alter wild codes rather quickly. The cost of editing data 
equals about forty percent of the job cost. On the average, this is about 
thirty to forty dollars. 

PROGRAM RUNS 

After the data are edited we first submit all the variables to a run 
on basic statistics which provides for means, standard deviations, and the 
range. An examination of the range often serves as a final check for wild 
codes. With this run we also request frequency distributions on any 
variables which will be subjected to transformations or receded into dummy 
variables. An examination of these basic statistics allows us to determine 
what variables to include in further analysis, how to transform variables, 
and what type of dummy coding will yield the best picture of a variable's 
relationship to other variables. For example, in a recent analysis of our 
R-300 follow-up data, we recede twenty-three out of the original seventy- 
five variables to allow for the computation of correlational statistics. 

Once the data is entered into a file, edited, and receding accomplish- 
ed, execution is rapid and inexpensive. Program runds account for about 
twenty-five to thirty percent of a project's costs; i.e., on the average 
of thirty dollars for small projects. For small data sets, thirty dollars 
devoted to statistical programs can generate just about every possible 
statistic one could need. But, as the data set grows, so does the cost. 

One must remember that even on simple runs the computer has to read 
all the data for each case in order to calculate the statistics. So, to 
submit an R-300 tape for analysis with all the clients for a fiscal year 
(56,000 records), costs anywhere from forty to sixty dollars for a single 
cross-tabulation. The author has seen a singly six-by-four cross-tabula- 
tion run on 200,000 cases cost one hundred fifty dollars. 

Therefore, we have decided to analyze small data sets and samples 
from the larger client population data base. The collection of statistics 
on the large client population will be left to our department computer 
which gives us basic statistics each year. Smaller samples of our data 
will be used when performing statistics such as factor analysis and mul- 
tiple regression. An example of our procedures in this latter area follows 
in the next section. 

AN EXAMPLE 

As part of the present paper, I wish to review a small project which 
we analyzed. This serves as an example of the experience we have gained 
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in data analysis over the recent months. Another evaluator* and I were 
asked to assist a Michigan rehabilitation facility in analyzing some of 
their program data. The facility delivered services for the rehabilita- 
tion of 86 severely disabled persons. This was part of a special project 
partially funded by a Bureau grant. We were anxious to use this small 
data set as a test trial of our new computer capacity. 

The Program Evaluation Unit was given a data entry document on each 
of the 86 clients. The data was a collection of demographic information 
on participants and some information on outcome after service. We were 
able to enter the data from the data sheets via the computer terminal. 
The aim of the analysis was to locate any client characteristics which 
related to program outcome; i.e., competitive employment, 

A. Client Characteristics 

The first step in the analysis was to examine the frequency 
distributions for client characteristics. The distributions for each 
data element are presented in Table 1. The basic picture presented 
by these data indicate clients who are in their early twenties, who 
have cerebral palsy, possess a high school education, have no work 
experience, and are living at home. 

Client status at program termination showed 30 percent of the 
clients referred to vocational services, followed by 13 percent of 
the clients on a medical hold. Approximately 12 percent of the clients 
were placed in competitive employment. 

B. Factor Analysis 

In order to place the variables into a factor analysis, the 
basic frequency distribution of the nineteen variables was examined. 
Eight variables were transformed into dummy variables. Additionally, 
seven variables of an interval nature were receded into class inter- 
vals. This latter transformation allowed for a more appropriate 
representation of the scale and for the calculation of an accurate 
mean. 

These transformations are of general interest since there is a 
great similarity between the way these variables were coded and the 
type of codes found on an R-300 data set. Many of these transforma- 
tions need to be carried out on the R-300 before any analysis can 
take place. Table 1 presents a list of the original codes and their 
transformations . 



*Data for this project were collected and analyzed by Geraldine 
Hansen, Ph.D., and the present author. 
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Now that all variables were in dummy or interval form, they 
could be placed in a factor analysis. The analysis utilized was a 
principal factor solution with commonalities placed in the diagonals 
of the correlation matrix. These factors were then rotated using 
the varimax method. The statistical package used was the University 
of Michigan OSIRIS IV. The solution generated ten factors of which 
the first five factors were interpretable. These factors are present- 
ed in Table 2. 

An inspection of Table 2 shows that only a few variables define 
each factor. Factors I and IV show that having a work history is 
related to one's ability to be mobile. The other factor of interest 
is the last factor, Factor V. Here we find that placement in com- 
petitive employment is related to length of stay in the program. 
Since only these two items load on this factor, no real interpretation 
can be given about the causal direction of this relationship. That is 
we cannot tell if successful outcome is "caused" by staying in the 
program for a longer time or if the ability to obtain a job helps one 
to also come to the program. 

C. Multiple Regression 

In an attempt to better understand the relationship among the 
variables and the goal of the program (i.e., employment), a multiple 
regression was performed. The variables were placed in this analysis 
using the outcome, competitive employment, as the dependent variable. 
The results of this analysis are presented in Table 3. 

In this analysis three variables relate positively to competitive 
employment. These were number of days in the program, drives own car, 
and able to take a bus. As in the factor analysis mobility was 
positively related to work. So, mobility was important in work 
history and outcome. As in the factor analysis length in the program 
was also related to outcome. When other variables are controlled, 
the program contributes 12 percent of unique variance to outcome. 
Thus, we can feel more confident that the program has a positive 
contribution to outcome. However, we are still unsure as to what 
this contribution is. 

Summarizing, from our factor analysis and multiple regression, 
we can see that the length of stay in the rehabilitation program is 
associated with outcome. The only client characteristic related to 
employment is mobility. The mobility of the client is related to 
both pre- and post-service employment. Mobility can be separated 
from the effect of the program, since the partial correlation with 
mobility controlled still leads to a relationship between outcome 
and length in the program. 
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TABLE 2 



FACTOR ANALYSIS OF AIDS PROJECT 





Factor Percent Variance 
Loading Accounted for 


Cumulative 
Percent 
Variance 


Factor 1 






Walk 
Bus 
Work History 


.83480 15.9 
.76343 
.57312 


15.9 


Factor 2 






Regular Education 
Onset of Disability 
(after birth) 


.84981 12.1 
.67947 


28.0 


Factor 3 






Age (older) 
Independent Living 
Situation 


.67947 9.7 
.75796 


37.8 


Factor 4 






Car 
Work Samples Passed 
Math Level 
Work History 


.81023 8.0 
.80029 
.54138 
.43721 


45.8 


Factor 5 






Days in Program 
Competitive Employment 


.70977 6.7 
.78821 


52.6 
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TABLE 3 



Variable 



REGRESSION ANALYSIS TO COMPETITIVE EMPLOYMENT 



Partial R R Squared T Ratio 



1. 


The greater number of 










days in the program 


.380 


.1270 


3.7307 


2. 


Drives own car 


.332 


.0932 


3.1959 


3. 


Able to take a bus 


.327 


.0901 


3,1418 



F-Ratio for the regression 9.203 
Multiple correlation coefficient .5019 
Explained variance .2519 
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Thus > the analysis of this special project not only demonstrated the 
value of the rehabilitation program but also demonstrated the value of 
our data analytic capacity. The expanded capacity with the University of 
Michigan computer allowed us to enter and retain evaluation data. We were 
then able to transform variables, run various analytical procedures, and 
discover relationships not known to us. 

CONCLUSION 

The Michigan Program Evaluation Unit has two alternatives for data 
analysis. The first is the production of routine program information from 
the Michigan Department of Education's departmental computer. The depart- 
ment's computer is used to generate reports which summarize data on clients 
and counselors on a quarterly and annual basis. These reports provide us 
with global parameters on the client population with breakdowns by sex, 
rehabilitation outcome, and diagnostic group. 

Recently the unit has developed a second data analytic capacity with 
the University of Michigan's Computer System. This system allows for the 
analysis of special projects. Special projects may be of the type in which 
data collected Independently of the Bureau's Management Information System 
are analyzed. The example analysis presented in this paper is of this 
type. Special projects may also include sampling case records from the 
Management Information System and placing these in an extended analysis. 

The combination of these two analytic capabilities has placed the 
Program Evaluation Unit in a good position to offer the Bureau's manage- 
ment better evaluation and consultation. 
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COMPUTER ACQUISITION AND OPERATION 
Mississippi Vocational Rehabilitation for the Blind 



Background 

The Rehabilitation Act of 1973 and subsequent amendments thereto have 
placed greater demands on rehabilitation agencies in terms of service pro- 
vision to clients and agency accountability for such services. Thus, like 
other rehabilitation agencies, Mississippi Vocational REhabilitation for 
the Blind (MVRB) experienced a sharp increase in the amount of raw data 
generated, routine reporting requirements and special reporting require- 
ments. MVRB soon realized that its manual record-keeping system could not 
keep pace with the increasing data output and began searching for an alter- 
native method which, because of cutbacks in personnel and budgetary 
constraints, had to be inexpensiv and capable of simple and immediate 
implementation. The answer was found in MVRB's sister agency, Vocational 
Rehabilitation-State Department of Education/Mississippi, which had been 
developing a system of computerized information retrieval since 1968. 
In the fall of 1973, the director of both agencies met and an agreement 
was reached by which MVRB would use the general agency's Client Information 
System, but without modifications, in exchange for the processing costs. 
In mid-1974, MVRB began collecting statistical data for FY-75 caseload 
reporting. In 1977, the agreement was expanded to include MVRB's use of 
the general agency's Case Service Expenditure System which permitted re- 
trieval of limited financial data. Under the conditions of the original 
expanded agreement, MVRB's involvement was strictly a hands-off role. 
Although this agreement allowed MVRB to meet its immediate needs, MVRB 
management was keenly aware that the time lapses in processing MVRB data 
(ten to twenty-eight days from collection of data to its return to MVRB 
district offices in a usable form) greatly negated the three (3) hallmarks 
of a sound information system: responsiveness, timeliness, accuracy. 

Assssment of 



With the award of a contract to establish a Model Evaluation Unit 
(MEU) , the Director of MVRB requested that the MEU analyze MVRB's data 
processing needs and explore possibilities for actualizing an internal 
data processing system. The MEU's Systems Analyst spearheaded the study 
of MVRB's existing Management Information System. Assistance was provided 
by the MVRB staff, the consultative services of Gehl and Associates 
(Atlanta, GA) , the consultative and technical services of RSA-Region IV 
Federal Office (Atlanta, GA) and the research and analytical services of 
the MEU staff. The study team identified the single most glaring 
deficiency to be the lack of access to available data in a form capable 
of supporting the decision-making process necessary for managerial 
efficiency and effectiveness. While instances of non-existent information 
as a deficiency were also identified, the overriding concern was the 
MVRB had no acceptable method of easily retrieving usable data. 
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The study team set the following criteria by which to judge alternative 
solutions to MVRB's data processing problems: (1) flexibility of the alter- 
native proposed; (2) accuracy of information via the alternative proposed; 
(3) cost of alternative implementation and maintenance; (4) method respons- 
iveness and (5) adaptability to change. Change as used here refers to 
possible future alterations in routine and special reporting mandates as 
the result of national and/or state programs and policies. Among the 
possible solutions proposed and considered were: (a) increase the existing 
staff to expand manual operation; (b) use microprocessor based flexible 
disk units to store some data; (c) use off-site batch processing; (d) use 
on-site batch processing; (e) use on-site data entry with off-site batch 
processing; and (f) use on-site data entry and batch processing. 

During the evaluation of these proposed alternatives, vendors of data 
processing equipment and systems specialists were contacted to secure their 
expert input. These vendors included IBM, Datapoint, IV-Phase, Telex, and 
Texas Instruments. Additional evaluative input was provided by the Central 
Data Processing Authority (CDPA)-State of Mississippi (Jackson, MS), Gehl 
and Associates (Atlanta, GA) , Mockbee, Edwards and Associates (Jackson, 
MS), South Central Bell Telephone Company (Jackson, MS), and managers of 
state agency electronic data processing units. Ultimately, the study team 
reached the consensus that MVRB's needs in data processing could best be 
served through acquisition of its own data processing equipment and the 
actualization of a staff capable of systems development exclusively 
responsive to MVRB's needs. 

Equipment Acquisition Procedures 

The first step in acquiring data processing equipment for MVRB was 
for the MEU's systems analyst to develop a Proposal for Data Processing 
System for approval by the MVRB Director and subsequent submission for 
approval to the CDPA-Mississippl and RSA-Washington, D.C. The proposal 
was prepared in accord with the CDPA's format as it is the agency in 
Mississippi which has authority statewide to receive, review, approve or 
reject all requests for data processing systems. The proposal prepared 
by the MEU's systems analyst included, briefly, the following: 

I. STATEMENT OF PURPOSE 

Stated findings of study team and proposed remedial action 
via acquisition of a data processing system for MVRB. 

II. BACKGROUND 

Explained what factors made it expedient that MVRB secure and 
assume management of a data processing system, how MVRB was 
currently handling its data processing needs, and justification 
to support MVRB's request. 
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III. SUMMARY OF NEED APPLICATIONS 

The proposal noted that at a minimum seven systems were needed: 

A. Case Recording and Expenditure System with the minimum 
capabilities of: 

1. Immediate data entry, validation and retrieval at the 
state office level; 

2. Prompt payment for vendor services as an automatic 
by-product of the system; 

3. Provision of caseload and budget reports for field 
and district staff on a current basis; 

4. Responsiveness to the changing needs of management and 
of the Federal reporting requirements; 

5. Provision for security and confidential information; 

6. Supporting fiscal accounting by maintaining records in 
an auditable manner; 

7. Supplying data needs for program evaluation, research 
and planning; and 

8. Alerting management at several levels of any condition 
which is unusual or in need of immediate attention. 

B- Payroll Information and Accounting System encompassing the 
following requirements: 

1. Data entry, editing, and retrieval by personnel staff 
as the need occurs; 

2. Automatic generation of all monthly payroll payments 
and required registers and reports. 

3. Automatic distribution of accounting information for 
state and Federal budgets; 

4. Generation of all quarterly and annual wage statements; 

5. Allocation of staff and fringe benefit costs by pro- 
gram, area, or district for cost-benefit analysis; and 

6. Responsiveness to special reporting requests, 

C. Administrative Accounting and Payments System designed to 
address: 

1. Ease of data entry, edit, and retrieval by accounting 
staff; 

2. Generation of vendor payments with required registers 
and distribution reports; 

3. Allocation of expenditures by program, area or dis- 
trict for cost-benefit analysis; 

4. Generation of projected costs for the budgeting 
process; and 

5. Line item comparison of budget versus actual costs 
incurred . 
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D. E l_quipmen . t_ I n ven t o T y^ T r a c kjji g System characterized by: 

1. Ease of data entry, edit and retrieval by accounting 
staff; 

2. Generation of budget information regarding new and 
replacement equipment; 

3. Allocation of equipment costs by program, area or 
district for cost-benefit analysis; and 

4. Detail information for conducting audits of equipment 
statewide. 

E. Business Enterprise Program (BEP) Information System 
capable of : 

1. Easy data entry, edit and retrieval by BEP staff; 

2. Tracking and reporting all financial transactions by 
stand location on a current basis; 

3. Providing for automatic generation of all required 
payroll statements and reports by location; 

4. Generating current Profit and Loss Statements by 
location; 

5. Providing immediate access to equipment inventory and 
repair history by location, description or ID number; 

6. Providing for the auditing of stock goods at each 
location by producing perpetual inventory reports; and 

7. Enhancing managerial decisions by providing current and 
accurate information for use at the statewide, area 
and local levels 

F. Management Information and Evaluation System designed to 
include the following capabilities : 

1. Provide state, area, and district caseload profiles 
for the purpose of allocating agency resources; 

2. Provide statistical analysis of case characteristics 

in order to evaluate and project program effectiveness; 

3. Provide data necessary to assign direct and indirect 
agency costs on an individual case basis for the pur- 
pose of evaluating and projecting program efficiency; 

4. Provide data necessary to ascertain normal bounds for 
duration of service provision and to indicate those 
instances which fall outside these parameters in order 
to assure program timeliness; 

5. Provide data necessary to summarize the financial 
standing of each MVRB component on a current basis, 
thereby preventing the wider-utilization of Federal 
allocations in the provision of services to clients; 

6. Provide for the project of line-item budget needs 
based on the most current actual and projected cost 
figures; 
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7. Provide storage and retrieval of data thus permitting 
the agency to respond quickly to requests for in- 
formation from Federal, state, private and public 
sources; and 

8. Reduce the volume of clerical responsibilities and 
release valuable manpower for higher levels of pro- 
fessional action. 

IV. ALTERNATIVES CONSIDERED 

Alternatives previously noted in this article were discussed 
in depth. 

V. LONG RANGE PLAN 

The CDPA-Mississippi required that along with a proposal for 
data processing, the agency submit a five-year plan which 
addressed the projected thrust of the agency in terms of data 
processing. The plan submitted included coverage of such areas 
as projected upgrades, applications, files, equipment, and the 
like. 

VI. ALTERNATIVE SELECTED 

Explained how the particular system configuration was selected. 
This section of the proposal cited the criteria used by the 
study team, the utilization of existing agency resources, the 
lack of increase in existing agency budgeted costs, the avail- 
ability of limited Federal funds, and the fact that no additional 
agency staff would be immediately required as factors impacting 
the selection of a system configuration. 

VII. SYSTEM SPECIFICATIONS 

The HVRB proposal listed (1) remote job entry, (2) data entry 
(3) GIGS, and (4) local processing as specifications, addressing 
each separately so that bids from vendors could be compared 
equally. (Advertising for bids on data processing equipment 
is handled by the CDPA-Mississippi which, in turn, advises the 
agency of the bidding outcome.) 

VIII. SOURCE OF FUNDS 

Contained a statement to the effect that funds for the acquisi- 
tion of the proposed system would come from Federal and state 
appropriations already budgeted, with no more than 15 percent 
of the system's cost being covered by state-appropriated funds, 

It should be noted here that the original proposal submitted to the 
CDPA had to be revised and re-submitted in accordance with the GDPA's 
interpretation of the needs of MVRB as described in the original proposal. 
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The Sys t^em In Opera t ion 

The culmination of more than a year's work came in January, 1980, 
when a stand-alone mini-computer was installed in the HVRB state office. 
The acquisition of this equipment signalled, in MVRB, the advent of a 
new methodology in the agency's design for providing rehabilitation services 
to Mississippi's blind and visually impaired population. 

As of July, 1980, MVRB's data processing system is under active 
development. Since all MVRB functions revolve around the activities of 
field or client service personnel and their interaction with clients, 
major efforts have been directed toward developing an on-line update and 
retrieval system of client statistical and financial data. When completed, 
this system will provide timely data for caseload management at all levels 
of state agency activities. Considerable effort has also been devoted to 
pre-planning systems and brainstorming with MVRB staff in an effort to 
identify major concerns and anticipate data requirements at each level. 

Beyond refining statistical and financial capabilities, there are 
plans to develop a flagging system to enhance efficiency in agency oper- 
ations. Such a flagging system would include timely client movement, 
timely client contact, Annual Review of IWRP, Review of Ineligibility 
Decision, and Review of Extended Employment in Facilities, to name but a 
few of the almost limitless possibilities, 



NOTE: This prescriptive article necessarily abbreviates certain steps in 
the acquisition and beginning operation of Mississippi Vocational 
Rehabilitation for the Blind's data processing system. Requests 
for specific details or answers to questions may be addressed to 
the Mississippi MEU at P. 0. Box 4872, Jackson, MS 39216. 
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WORD PROCESSING APPLICATIONS OF 
THE OREGON MODEL EVALUATION UNIT 

Lynn Delia, Laurilee Hatcher and Ross T. Moran 
State of Oregon Department of Human Resources 

When the Oregon Vocational Rehabilitation Division (VRD) first con- 
templated involvement with the model evaluation unit/management informa- 
tion support (MEU/MIS) project, we were concerned about the anticipated 
increase in secretarial/clerical paperwork which would result from contract 
requirments. Because of this concern and our desire to build program 
evaluation/policy support capacity, we requested the leasing of a word 
processing (WP) system in our contract proposal. 

After a comprehensive needs assessment of potential system users and 
a comparative review of the available technologies, we obtained a CPT 8000 
system. The system was installed in October, 1979. 

We have developed applications for the word processing system to 
handle work derived directly from the MEU/MIS project, work associated with 
the unit activities distinct from the MEU/MIS project, and communications 
applications. This paper discusses each of our applications of the word 
process which have been implemented and some of the future applications 
which we anticipate. 

Merging lists of names with letters/envelopes for mall surveys. Con- 
siderable use of the WP capability to sort and merge has been, made by the 
MEU/MIS in preparing lists, deletions and labels for mailed surveys. This 
was especially useful in meeting the requirements of pretesting the pro- 
posed VR evaluation standards which involved production of personalized 
follow-up surveys and reminders which were sent to clients. 

Issue Tracking System. An outgrowth of the MEU/NIS contract was the 
identification of agency policy and procedure issues of which the Adminis- 
trator wished to be aware. After the issues were defined and a reporting 
format designed, the tracking system was input to the WP system. Updated 
monthly by additions, modifications, and deletions, the tracking process 
is ideally suited to the capabilities of WP. An article in an earlier 
issue of the MSOA describes the issue tracking system in greater detail. 

Service Request Log. The agency has long followed the practice of 
documenting requests for management Information. For many years these 
requests were maintained and logged in a handwritten system. Acquisition 
of the WP offered the opportunity for electronic logging. The WP log was 
established beginning with outstanding requests. As new requests were 
produced, new lines were added to the log. It is possible to produce lists 
of requests by numerical order, requester, data requestd, date wanted, 
priority status, person assigned to track, or date completed. 
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MEU/MIS Project Work Plan. Soon after the initial contract award, the 
MEU/MIS staff determined the necessity of an easily revised "project work 
plan." This was directly related to the contract requirement to develop 
an "operation tracking system." A format was developed and implemented. 
This first tracking system was found by the unit to be less useful than 
hoped, and was significantly revised near the end of the second project 
year. The new format is briefer and much more easily updated and copied. 
Using the sort/merge capability of the machine, it is also now possible 
to produce separate lists of only those tasks assigned to specific 
individuals. 

Structuring and^cpmpleting both regular and jpecial statistical 
reports . The primary WP features employed in statistical report genera- 
tion are the vertical line capability and the automatic decimal alignment 
capabilities of the system's 10-key pad. The vertical line allows creation 
of boxes, graphs, charts, etc. The automatic decimal alignment provides 
for speedy input of numbers of varying lengths by maintaining the integrity 
of the "ones" column or, where applicable, a decimal point. The time- 
saving element of these features is well appreciated when preparing a 
report such as the agency's budget document, where time is of the essence. 

Math Pak. Also a new acquisition, the recently developed CPT "Math 
Pak" software allows the WP operator to perform simple and complex 
mathematical functions. Many calculations which can be completed on a 
standard calculator may be achieved using the WP keyboard, from simple 
arithmetic to use of memory and square roots. OVKD has just begun identi- 
fying and developing applications for this feature above and beyond simple 
verification of statistical reports. 

BASIC, The CPT 8000 also accepts software which converts it from a 
word processor to a mini computer, programmable in the computer language 
BASIC. We have recently acquired this capability. We expect that it will 
fill the technology gap between the capabilities of our statistical hand 
calculator and the large computer we share with our umbrella agency. 

Optical Character Recognition (OCR) . Having trained seven staff 
members as WP operators, it was obvious that not all could use the system 
simultaneously. Consultation with the vendor representatives introduced 
the concept of optical character recognition (OCR) to the agency. OCR 
allows materials typed on a regular typewriter to be machine read. Ar- 
rangements were made for OVRD staff to utilize the OCR reader at another 
State agency during designated hours. OCR typing elements were purchased 
for agency typewriters and the input capability has multiplied. By work- 
ing with the trained operators, secretarial staff untrained in WP have 
become able to produce high quality originals and revisions in a timely 
manner. At the current time, however, due to cost, OVRD has no plans to 
acquire its own OCR reader. 

Diskette storage of various administrative policy and procedure 
manuals so they may be easily updated. At the time the WP proposal was 
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written it was anticipated that the agency's 1,300 page administrative 
manual would be entered into the WP system. Subsequently, the decision 
was made to revise the manual in its entirety. Drafts for the revised 
manual are now typed for OCR input. 

Administrative Reviews. The agency's Administrative Review staff 
evaluates the quality of field case work. They produce reports following 
each review. These reports are about 20 pages long and are substantially 
revised at least once. Agency secretarial staff have been able to make 
more efficient use of their time by using the WP for these reports. 

Client Equipment Repossession List. The client equipment reposses- 
sion list, which is sent to field offices on a monthly basis, is similar 
to the Issue Tracking System in its applicability to WP. It identifies 
equipment purchased by the agency to facilitate clients' rehabilitation. 
When a client's case is closed as other than "rehabilitated," this equip- 
ment is repossessed by the agency and becomes available for transfer from 
one field office to another. This reduces the purchase of duplicate 
equipment. WP use for this list began in June 1980, and appears to be 
functioning smoothly. 

Creation of a document retrieval system for the agency library. We 
originally planned to use the WP system as a form of automated "card 
catalogue" for the documents in the agency's small library of monographs, 
reports, books, and other literature. Some form of "coordinate indexing" 
system of documentation was planned, which would be well-suited to the 
capabilities of the word processor. This plan was superceded by a scheme 
whereby we transfered the bulk of our documents to the state library. It 
was also thought that with the communications capability of the system we 
could tie into a national or regional computerized bibliographic system 
to "expand" our access to relevant documents. Several contracts have been 
made regarding this experimental work which has demonstrated the feasibility 
of this and we anticipate a more extensive implementation of this concept 
in the third project year. 

Acoustical Coupler. Peripheral to the basic WP system, the acoustic- 
al coupler is a new acquisition, and not yet much used. It allows "com- 
munication" between the OVRD WP and any other WP system or computer which 
also has a coupler and compatible "protocol" (selectronic send/receive 
rates, language, etc.). Currently, the MEU/MIS has established a communi- 
cations relationship with the WVRTC computer. This paper was transmitted 
from Oregon to WVRTC via the coupler for output through WVRTC 's phototype- 
setter (also connected to the computer there). We will also use this 
feature to access the national bibliographic systems discussed above. 

Organizational Charts. The agency is required by the State to main- 
tain up-to-date organizational charts . Updating these by hand or 
traditional typewriter is time-consuming and often leads to xeroxes of 
xeroxes of xeroxes. Early in 1980 the agency's charts were put into the 
WP system. The system's capability to produce high quality originals was 
very well received. 
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As the agency reorganized in June 1980, the system's capability in 
the area of organizational charts proved to be a real asset, since as many 
as five complex reorganization drafts were produced before the final was 
accepted. It was possible to complete, copy and deliver a revision in 
about a half-hour. 

Unit corr&5p^qn.dence* Initially, the system was employed for routine 
letters and memorandums. However, these types of documents are not often 
subject to revision* As the variety of applications increased, it became 
apparent that using the system for routine correspondence was not efficient, 

Trained operators from other agency units have utilized the system for 
merging standard correspondence text with variable names and addresses for 
applications such as information memorandums and meeting notices. 

One agency product proposed for WP is the Personnel Unit's "Staffing 
Allocation /' a somewhat complex listing of agency positions, employees, 
EEOC codes, classification, position number, etc. This list is about 12 
pages in length and must be revised monthly. Its format is ideally suited 
to the WP application. 

To make more efficient use of the WP, serious consideration is being 
given to implementing a double shift. With more extensive use of the CPT 
8000 through initiation of bibliographic searches and its use as a mini- 
computer, the possibility of multi-shift operation of the machine has 
almost become a certainty. 

We have found many applications for the word processor in enhancing 
the capabilities of the MEU. While our early experience has centered on 
its use as a text editor, we are rapidly expanding its utilization so that 
it will truly function as an information processor capable of supporting 
and enhancing the program evaluation and management information functions 
of the MEU/MIS unit. 
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EVALUATION SECTION LIBRARY 

Leah Kuhns, Administrative Assistant 
Pennsylvania Bureau of Vocational Rehabilitation 



The Pennsylvania Bureau of Vocational Rehabilitation (BVR) estab- 
lished its Evaluation Section Library as the result of its Federal contract 
to develop "A Comprehensive State VR Program and Policy System Through a 
Model Evaluation/Management Information Support Unit." This contract man- 
dates that a library dealing with program evaluation be established, 

In the beginning all that was envisioned by Pennsylvania's Model Evalu- 
ation Unit (MEU) was a small library dealing solely with evaluation subject 
matter. But as the work progressed it became apparent that there was a 
need for a more extensive library which would serve the entire Bureau, and 
the library that was established, while known as the Evaluation Section 
Library, is, in fact, a BVR library. In addition to its program evaluation 
(PE) holdings it contains material in many areas of vocational rehabilita- 
tion (VR) , Including new rehabilitation trends, current legislation, specific 
disabilities, assistive devices, material for civic groups, etc. This 
library is now housed in the Central Office of the Pennsylvania BVR. and 
serves BVR 1 s 15 District Offices, four Regional Offices, the Hiram G. 
Andrews Center, (a 500 client rehabilitation center operated by the BVR) 
and three Disability Determination Offices. 

The creation and subsequent development of the Library was not an 
easy task. It required careful planning, much research and a lot of hard 
work on the part of the staff of the BVR Evaluation Section. But we feel 
the end result was well worth the effort, and we would like to share with 
you the process by which the Library was established and to describe its 
operation now. 

Since this was a totally new undertaking in an unknown area, there 
was an obvious need for expert advice and assistance. This was sought 
from many and varied sources. One of the more evident needs was for the 
expertise of professionals in established library systems. The Pennsyl- 
vania State Library, located in Harrisburg, was solicited for technical 
assistance on the establishment, maintenance and continuing development 
of a library. In addition, its staff provided information on the various 
activities and resources within the State Library which would enable us 
to supply the BVR field offices with pertinent material to meet their 
needs. For example, they provided us with a list of all periodicals re- 
ceived by the Pennsylvania State Library and placed us on their mailing 
list to receive notification of all new acquisitions. In addition, the 
Evaluation Section Technical Assistance Center (TAG) the official name 
o the Evaluation Section Library has been listed in the Pennsylvania 
State Library's publication "Directory of Libraries serving the Govern- 
ment of the Commonwealth of Pennsylvania." This directory shows the 
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number and types of information resources available to the various 
agencies, departments, commissions, and boards of the Commonwealth of 
P ennsylvania . 

We had another very important resource in the library which is located 
at the Hiram G. Andrews Center in Johnstown, PA. The Center Library has a 
section of material available for use by Center clients and a separate 
section of reference materials for staff use. In addition, it is in the 
process of establishing a medical library for use of the Center staff. A 
two day visit to this Center produced valuable information as to cataloguing 
procedures, suggestions about periodicals and books to be acquired, pub- 
lishers, other types of resources. We continue to receive assistance from 
this library when conducting research and have need of materials not con- 
tained in our TAG. 

We also contacted and received much valuable assistance from many 
sources outside Pennsylvania. Massachusetts, Minnesota, Ohio, Oregon and 
Virginia all have state vocational rehabilitation agency libraries, the 
staffs of which shared their expertise with us. Two members visited the 
National Rehabilitation Information Center (NARIC) in Washington, D.C., to 
obtain its assistance in dissemination, utilization and library science 
techniques and to discuss the purpose of NARIC and how it serves rehabili- 
tation professionals. Personal visits were also made to the President's 
Committee on Employment of the Handicapped, the Office of Handicapped 
Individuals, the Office of Civil Rights, and Project Share, all of which 
provided various types of information and assistance. (Project Share, 
located in Germantown, MD, is a free-of-charge, on-demand service which 
permits organizations and individuals to contact its clearinghouse by 
mail or telephone with requests for information on various topics . The 
responses to reference requests are customized, annotated bibliographies, 
generated primarily from the Project Share data base.) Two persons 
attended the first meeting of the Rehabilitation Information Round Table 
in Washington, D. C., an organization designed to promote the sharing and 
dissemination of rehabilitation information, and working relationships were 
established with ICD in New York and the Oklahoma State University Clearing- 
house in Stillwater, OK. These agencies assist our library in locating 
material on subjects being researched. For instance, the TAG received a 
request from the field for a film to show persons who are being deinsti- 
tutionalized from a state mental hospital to let them know what VR is and 
how it can assist them. Several films were located at the Oklahoma State 
Clearinghouse which sent them to the TAG for review. As a result, two 
films are being purchased and will be made available to field personnel. 

Another phase of the groundwork was contact with publishers, numerous 
rehabilitation-related organizations and other sources that have available 
VR and PE materials which could be acquired for inclusion in the library. 

The actual creation of the library required many decisions in three 
principal areas physical needs, acquisition of materials, and operations. 
Foremost among the physical needs was housing a suitable location where 
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people would feel free to visit and browse was selected. Then it was 
necessary to order bookcases, book covers, magazine racks, a filing 
cabinet, and other needed equipment. Selection of material to be ordered 
was done by a three-person review team which carefully selected material 
on the basis of the audience we wished to serve. Among other things, a 
$200 deposit account was established with NARIC to receive photo duplica- 
tion of available documents as needed. (An unexpected but most welcome 
source of materials was and continues to be persons working in our agency 
having personal holdings which they donated to the Library.) In operations, 
a most Important consideration was the choice of a cataloguing system. 
A lot of study went into its selection. Various cataloguing systems were 
researched, numerous thesauruses obtained, and the development of our own 
system considered. In the end, it was decided to adopt the classification 
designed in 1961 by the Oregon Division of Vocational Rehabilitation. This 
decision was based primarily on the fact that it was a proven system which 
had been adopted with success by various agencies which house rehabilita- 
tion literature. Providing detail and at the same time allowing for 
flexibility to meet individual needs, it has proven to be a very usable 
system in our library. Its classification system identifies nine major 
subject areas and these are subdivided to allow for the classification of 
publications which deal with various aspects of these major subjects. The 
major subject areas and numbers assigned are: 100 series Rehabilitation; 
200 series Disabled and Handicapped Groups; 300 series Impairments; 
400 series Rehabilitation, Cooperation and Integration; 500 series 
Rehabilitation, Financing; 600 series Social Problems; 700 series-- 
Government; 800 series Administration; and 900 series Reference Works 
not Related to the Above Subjects. 

When a publication is received for inclusion in the library, it is 
first classified and cataloged. For books, pamphlets, and miscellaneous 
material, two 3" x 5" cards are prepared: one by title and one by author. 
The publication is then reviewed and additional cards are prepared under 
subject headings. (The number of subject cards is determined by the 
number of main subjects covered in the particular publication.) The 
material is then placed on the shelves or in the vertical file, which is 
a file cabinet containing magazine articles, pamphlets, and other 
material which it is not practical to store on bookshelves. Material so 
housed is identified in the card catalog with a "VF" under the classifi- 
cation number. 

A separate card file was established for periodicals, of which we 
are presently receiving approximately 75. In order to maintain an 
accurate up-to-date record of each periodical received, a card is 
prepared for each periodical. As each new issue is received, it is 
recorded on the card and circulated among the Evaluation Section staff. 
Upon completion of circulation, the periodical is returned to the library 
where it is placed in a wood-grain plastic holder with other issues of the 
same periodical. All periodicals are located together in a section of 
the library which is easily accessible to anyone wishing to use them. 



33 



Since the main purpose of the Evaluation Section Library is to provide 
pertinent information to all Bureau of Vocational Rehabilitation personnel, 
a method for dissemination of material needed to be established. The tool 
selected to assist in the dissemination of material was the "Rehab Brief," 
a biweekly publication prepared by the Rehabilitation Research Institute 
of the University of Florida, located in Gainesville. Since all field 
personnel receive this publication, it was decided to include an insert 
with each "Rehab Brief" making its readers aware of Evaluation Section 
Library acquisitions. Each issue of the "Rehab Brief" deals with a 
specific subject of interest to rehabilitation personnel. Therefore, 
the basic format for the insert is a short one-paragraph synopsis of the 
subject discussed in that issue. This is followed by a listing of material 
which is available on this subject in the Evaluation Section Library. 
The remainder of the two page insert is used to offer to the field other 
material which might be of interest to them. The material includes 
articles on various subjects, free material they can order from clearing- 
houses and other agencies, and any other information we feel would be 
beneficial. 

Since the main thrust of the insert is to obtain user involvement and 
the dissemination of material down to the counselor level, all personnel 
receiving this insert may call the Evaluation Section Library directly for 
information. This is an important facet of the Evaluation Section Library 
since it is a direct linkage with every level of BVR personnel. In 
addition, another important aspect of user involvement is obtaining feed- 
back directly from the field. Field staff are encouraged to contact the 
Library with suggestions for improvement, recommendations for acquisitions 
they feel would be beneficial for inclusion in the Evaluation Section 
Library and any other areas they feel would assist us. 

Technical assistance is an additional service which the Evaluation 
Section Library provides to persons requesting specific information. 
Thus far technical assistance has been provided on a wide range of 
subjects, a few of which are clothing adaptations for specific disabili- 
ties, specialized eating equipment, information on specific disabilities, 
accessibility design and functional limitation. It must be stressed that 
providing this type of assistance requires a considerable amount of time. 
To thoroughly research a request, all publications pertinent to this 
subject which are part of the Evaluation Section Libary are reviewed for 
possible inclusion. In addition, if we do not have sufficient, in- 
formation, the request is researched through the Pennsylvania State 
Library, NARIC, Research and Training Centers, RRRIs, and any other source 
we have knowledge of where this material might be available. 

Within the last several months, a loan service has been generated 
which makes available to the field various publications which can be 
loaned for a two-week period. Each issue of the "Rehab Brief Insert" 
lists new books available through the loan service. Since it is not 
unusual for more than one person to request each publication, a waiting 
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list has been established and books are loaned on a first-come basis. 
Accompanying each book is a memorandum stating this publication is being 
furnished on a loan basis and the due date for it to be returned. 

Within the next year a separate listing of all periodicals and books 
within the Evaluation Section Library will be prepared for dissemination. 

We have attempted to sumarize for you how we in Pennsylvania estab- 
lished the Evaluation Section Library, and how it now operates. Should 
you begin a project such as this, we feel the major emphasis must be on 
taking steps which would encourage user involvement. This is extremely 
important in order to obtain maximum utilization. 

There has been a great deal of satisfaction derived from the estab- 
lishment of the Evaluation Section Library, especially when we see the 
interest generated from the field staff. However, we feel it is important 
to stress the amount of time and effort needed to produce these results. 
It is by no means a small undertaking. In addition to maintaining the 
actual day-to-day operation of the Evaluation Section Libary, which in- 
cludes ordering publications, cataloguing material, and all other aspects 
of library work, we received approximately 150 requests from the field 
staff during the first year of operation. The 150 requests included 
information offered in the "Rehab Brief Insert" and requests from district 
offices and central office for specific technical assistance. Examples of 
technical assistance included information for presentations to civic 
groups on affirmative action for the handicapped, job readiness programs 
for the deaf, current legislation, and specific information on disabili- 
ties and assistive devices. We do not wish to discourage you from such 
an undertaking, in fact we wish to do just the oppositebut we do want to 
make you aware of the effort involved. 

We hope this information will assist you. If you have any questions, 
or want more information, please contact Leah Kuhns at 717-787-7312. The 
mailing address is Bureau of Vocational Rehabilitation, 1318 Labor and 
Industry Building, 7th and Forster Streets, Harrisburg, PA 17120. 
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PLANNING FOR AN INDEPENDENT LIVING PROGRAM IN VIRGINIA 
Stanley E, Portny and Associates, Inc. 

Title VII of the Rehabilitation, Comprehensive Services and Disabili- 
ties Amendments of 1978 (P.L. 95-602) authorizes several new program 
initiatives for the provision of comprehensive services for independent 
living which are designed to assist severely disabled individuals to 
function more independently within their homes, families and communities. 
As the state agency responsible for the coordination of programs and ser- 
vices to handicapped people in Virginia, the Department of Rehabilitative 
Services (DRS) is currently reviewing the existing activities designed to 
serve this target population and formulating integrated strategies for 
the design and Implementation of an Independent Living Program, 

To facilitate statewide planning for Independent living, DRS has award- 
ed a contract to Stanley E. Portny and Associates, Inc. (SEPA) , to evaluate 
the current system for delivering comprehensive rehabilitation services for 
independent living and to develop recommendations for its expansion and 
improvement. Specific study objectives are: 

To describe and develop estimates of the target population of 
severely disabled individuals in Virginia; 

* To estimate the need for comprehensive rehabilitation services for 
independent living in Virginia; 

a To determine and describe resources available to assist in meeting 
this need; 

9 To describe and evaluate the current program and service delivery 
system for providing these services in Virginia; and 

To develop recommendations for planning, managing and evaluating 
an Independent living rehabilitation program in Virginia. 

This project was initiated in January 1980, and completion is expect- 
ed in October 1980. The results of the study will be presented in a 
series of working papers and reports, the highlights of which are as 
follows: 

CONCEPTUAL FRAMEWORK - A WORKING PAPER 

Provides an introduction to the study background and approach; sum- 
marizes general concepts of independent living, including underlying 
principles and independent living terminology; reviews Federal require- 
ments for independent living programs; and discusses the preliminary 
planning for independent living which has been Initiated in Virginia. 
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TAXONOMY OF HUMAN SERVICES FOR INDEPENDENT LIVING - A WORKING PAPER 

Identifies and defines the different independent living services; 
identifies specific programs established under P.L. 95-602 which provide 
these services; identifies other state/Federal human service programs 
which provide services identical or similar to comprehensive services for 
independent living; analyzes the existing service delivery mechanisms and 
existing or potential service gaps. 

FINANCIAL SUPPORT AND ASSISTANCE FOR INDEPENDENT LIVING SERVICES 

Discusses different types of funding sources and mechanisms and the 
use of similar benefits; identifies different Federal and state resources 
available in Virginia; and presents detailed analyses of funding sources 
available for housing and residential services, transportation services, 
and attendant care. 

ESTIMATES OF THE DISABLED POPULATION IN VIRGINIA 

Discusses conceptual and methodological problems in estimating the 
severely disabled population; reviews ways in which population data may be 
used in planning; summarizes secondary data sources on the incidence and 
prevalence of disabling conditions in Virginia; and presents estimates of 
the disabled population in Virginia. 

ESTIMATES OF THE NEED FOR COMPREHENSIVE REHABILITATION SERVICES FOR 
INDEPENDENT LIVING IN VIRGINIA 

Identifies those independent living services considered to be of 
greatest Importance in Virginia; presents estimates of the general need 
for these priority services; and prioritizes the populations to be pro- 
vided these services in Virginia, 

PLANNING RECOMMENDATIONS AND STRATEGIES FOR AN INDEPENDENT LIVING PROGRAM 
IN VIRGINIA 

Identifies key independent living planning and implementation issues; 
reviews relevant study information; analyzes alternative planning options; 
and provides suggestions for additional studies to address identified 
information requirements. 

STUDY FINAL REPORT 

Provides a summary of the study and highlights of activities conducted 

As indicated, the Conceptual Framework was prepared to present his- 
torical and legislative perspectives of the intent behind the independent 
living movement, to provide a general orientation to the study and to 
establish a structure for the design and performance of subsequent study 
tasks. In the remainder of this article, highlights of the topics dis- 
cussed and issues raised in the Framework are presented. 
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HIGHLIGHTS OF THE STUDY CONCEPTUAL FRAMEWORK 

During the past three decades, several different concepts of independ- 
ent living have emerged; and the influence of each is reflected in the 
provisions of title VII of P.L. 95-602. According to the traditional 
concept , aids to daily living would be provided and physical modifications 
to the environment would be made to assist severely physically handicapped 
people to attend to their personal care and basic survival needs. The 
principal objective of services provided in accordance with this "medical 
model" was to enable handicapped people to live in non- institutional 
settings. 

At the start of the 1960's, the early vocational rehabilitation con- 
cepj: emerged, which viewed independent living rehabilitation services as 
vocational rehabilitation services which were provided to severely handi- 
capped persons who did not have a vocational goal. In addition to 
recommending that the independent living rehabilitation program be 
administered separately from the title I program, these early proposals 
included: 

e An expansion of the traditional concept to include both 
physically and mentally handicapped people; 

An expansion in the socpe of independent living services to be 
provided; and 

e An extension of the independent living goal from simply prevent- 
ing institutional care to improving the lifestyle of severely 
handicapped individuals. 

During the same time that efforts were being made to amend the 
Vocational Rehabilitation legislation in the 1960's, a concept of in- 
dependent living was being developed by handicapped people and the 
societal changes needed for them to participate fully in community life. 
The consumer concept , which was promoted largely by physically handicapped 
persons, emphasized a total integrated service system which included not 
only vocational rehabilitation, but other needed services as well. Central 
to this concept, which was based upon a self-help framework, were the 
issues of consumer sovereignty, self reliance and the political and 
economic rights of disabled people. 

The con t emporary rehab il it at ion cqncejxt , as reflected in the provi- 
sions of title VII of P.L. 95-603, is an amalgamation of aspects of the 
traditional, early vocational rehabilitation and consumer concepts. This 
concept consolidates in a single state-agency-administered program; 

o A concern for the independent living needs of both physically 
and mentally handicapped people, including children of preschool 
age and older blind individuals; 
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& An extension of allowable "comprehensive services for independent 
living" to include vocational rehabilitation services under title I, 
and any other services which enhance the ability for an individual 
to live independently and function within the family and Che 
community; 

A service delivery system which, in addition to the public state 
agency, may include: 

- other local public agencies 

- private, non-profit organizations 

- public or private non-profit "centers for Independent living" 
which are managed and directed by disabled people; 

e A substantial role for disabled people in developing the state 
independent living plan; and 

a A separate "protection and advocacy system" established by the states 
to insure protection of the rights of handicapped individuals re- 
ceiving independent living services. 

Of particular importance is the different focus on the source of prob- 
lems and methods for their solution in the vocational rehabilitation and 
Independent living philosophies. The vocational rehabilitation approach 
considers the problem to consist of an Individual's physical or mental 
impairment which reduces his or her vocational potential, and the solution 
to entail intervention by a rehabilitation professional to improve the 
client's physical and/or mental capabilities which ultimately will help 
the client to obtain gainful employment. The independent living perspec- 
tive, on the other hand, views the problem to reside in the dependence of 
the disabled person on care givers and the environment, and the solution 
to entail instruction in the techniques of self-help and advocacy to 
remove environmental barriers which ultimately will enable the disabled 
Individual to live more independently. 

Based upon the provisions of title VII and the concerns inherent in 
the evolution of the independent living movement, it is suggested that 
the Independent Living Program in Virginia should embody the following 
principles: 

e The severely disabled population in need of independent living 
services will be heterogeneous and will have diverse service 
needs. 

There should be appropriate consumer involvement in all aspects of 
an independent living rehabilitation program. 

& The Independent Living Rehabilitation Program should be designed 
to include a diversity of service delivery models. 
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o The Independent Living Rehabilitation Program should be organized 
to respond to individualized client goals. 

s There should be a clear distinction between the title I and 

title VII programs, and operational relationships and mechanisms 
for cross-referral should be clearly defined. 

The Independent Living Rehabilitation Program should be designed to 
help maximize the severely disabled persons ability to function 
within the existing system of available services and resources. 

o The Independent Living Rehabilitation Program should be organized 
to utilize fully similar benefits. 

o The limited availability of funding for title VII in the near 
future should be recognized and considered in the development of 
Virginia's program and funding plans. 

Title VII of P.L. 95-602 is comprised of five separate parts. 
Part A, "Comprehensive Services," authorizes grants to states for the 
provision of comprehensive rehabilitation services to individuals whose 
disabilities are so severe that they do not presently have the potential 
for employment but may benefit from vocational rehabilitation services 
which will enable them to live and function independently. Funds for 
Part A grants will be distributed to states on a 90 percent Federal match, 
with the amount of Federal funds to which each state is entitled being 
proportional to the relative size of the state's population, but in no 
instance less than $200,000. 

Part B, "Centers for Independent Living," provides the authority for 
states to establish Centers for Independent Living, either directly or 
through contracts with public or non-profit agencies. Part C, "Independ- 
ent Living Services for Older Blind Individuals," authorizes the provision 
of grants to states for the provision of independent living services to 
older blind individuals, to help them to adjust to their blindness by 
becoming more able to care for individual needs. Part D, "General Provi- 
sions," authorizes grants to states for the establishment of protection 
and advocacy systems to insure the rights of disabled individuals served 
under title VII take affirmative action to employ and advance in employ- 
ment qualified handicapped individuals. Part E, "Authorizations," 
authorizes funds for carrying out parts A, B, C, and D. 

At the present time, only funds under Part B have been made available 
for award to states. To qualify for receipt of these funds, a state must 
submit a plan as defined for Part A; however, no funds currently are 
available for Part A grants to implement programs described in these Part 
A plans. In addition to the legislation itself, the only other authori- 
tative policy statement which has been issued to date regarding the 
requirements for title VII programs is the proposed regulations (44 CFR 
231, November 29, 1979). 
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According to the legislation, the state agency is allowed consider- 
able latitude in establishing its Part A program. The general purpose of 
title VII is framed in broad terminology which may be interpreted to 
include a wide range of service needs and allowable service goals; 
examples of services which may be provided are included, but no require- 
ment to offer some or all of these services is made; and the requirement 
to submit a State Plan is included, but all choices regarding the services 
to be provided and the service delivery mechanism(s) to be used are left 
to the discretion of the state. Though the proposed regulations introduce 
additional clarifications of certain service definitions, eligibility 
requirements and eligibility determination procedures, the major discretion 
for determining the specific operational details of the Part A program is 
left with the state agency. 

The procedural requirements for the establishment of a Center for 
Independent Living under Part B are specifically enunciated in the legis- 
lation; however, again, the state agency is allowed considerable latitude 
in establishing its Part A program. The general purpose of title VII is 
framed in broad terminology which may be interpreted to include a wide 
range of service needs and allowable service goals; examples of services 
which may be provided are included, but no requirement to submit a State 
Plan is included, but all choices regarding the services to be provided 
and the service delivery mechanism(s) to be used are left to the dis- 
cretion of the state. Though the proposed regulations introduce addition- 
al clarifications of certain service definitions, eligibility requirements 
and eligibility determination procedures, the major discretion for deter- 
mining the specific operational details of the Part A program is left with 
the state agency. 

The procedural requirements for the establishment of a Center for 
Independent Living under Part B are specifically enunciated in the legis- 
lation; however, again, the state agency is allowed considerable dis- 
cretion regarding the target groups to be served and the services to be 
provided. An, though the proposed regulations stipulate that Centers 
should provide as many of the identified independent living services as 
possible, the latitude which the state agency has in establishing its 
Centers is still considerable. 

Though Virginia has submitted an Interim State Plan under Part A, the 
following actions related to Federal program requirements still remain to 
be taken in the near future: 

a Define and specify the pruposes, goals, and objectives of the 
overall Virginia ILR program and its components; 

Specify the quality, scope and extent of independent living 
services to be provided; 

a Determine an operational definition of the target population (s) to 
be served, including an order of selection; 



e Determine and specify the distinction between title I VR and 
title VIII ILR programs and develop plans for the organizational 
relationships and working arrangements between the programs; 

o Strengthen the organization of program policy consultation with 
handicapped consumers, provider groups and others; 

a Develop a plan for the utilization of other agencies, organizations 
and facilities in the provision of ILR services; 

o Develop special plans (where appropriate) for the provision of 
technical assistance to urban and rural poverty areas; and 

a Develop goals and plans for the establishment and operation of 
Independent Living Centers. 

In addition to these Federal program and planning requirements, the 
following state requirements must be reflected in overall department 
planning: 

& Coordination of ILR planning with the reorganization of structure 
or administrative procedures in response to the "local option" 
initiative under Section 15.1-36.2, Code of Virginia. 

a Coordination of activities of the ILR program with MH/MR Chapter 10 
Boards; 

e Development of plans for coordination of the ILR program with the 
Community Rehabilitation Service System (CRSS) ; 

Development of plans for Interagency coordination and linkage 
between the DRS program ILR and Development Disabilities, Depart- 
ment of Health, Department of Mental Health and Mental Retardation, 
Department of Welfare, Virginia Commission for the Visually 
Handicapped, Virginia Employment Commission, Virginia Council for 
the Deaf, Virginia Housing Development Authority, Office of the 
Secretary of Transportation, and other appropriate Virginia state 
agencies; and 

a Coordination of plans for the ILR program with the Departmental 
Management and Planning System Objectives (MAPS) and with state 
budget constraints. 

Additionally, some of the major Part B Federal planning requirements 
which should be addressed entail: 

Development of a plan to involve handicapped individuals in the 
policy direction and management of the Center and in employment 
within the Center; 
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Development of plans to insure that the Center offers a combination 
of the specified ILR services; 

Determination of the group (s) of handicapped individuals to be 
served by the Center; 

Planning and obtaining of commitments for matching funds for the 
Center; 

Specification of the relationship of the Center to the overall ILR 
program. 
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A SYSTEMATIC APPROACH TO TRAINING NEEDS ASSESSMENT 
FOR VOCATIONAL REHABILITATION PROGRAM EVALUATORS 

Jeffrey K. Messing, Ed.D,, West Virginia University 
Richard A. Nida, Ph.D., West Virginia Research and Training Center 

The assessment of new skill and knowledge needs for human service 
workers is critical in a period of increasing demand for outcome evalua- 
tion, decreasing funds, changing job expectations and future opportunities. 
Not only must human service workers have the capacity to work with clien- 
tele and programs experiencing these pressures, but they must also be 
able to respond to their own personal situations as they experience 
increased demands for more effective and diverse functioning. Increased 
functional requests necessitate the development of a new repertoire of 
skills or a reorganization of existing skills. In today's world of re- 
habilitation where budgets are decreasing and case service, personnel and 
administrative costs continually are rising, the amount of funds allocated 
for refurbishing, redesigning or replenishing existing skill levels are 
severely limited. Therefore, it is imperative to accurately assess indi- 
vidual staff training needs and to effectively streamline programs to meet 
these needs most cost-effectively. 

This article presents and discusses a Staff Training Needs Assessment 
(STNA) strategy which focuses on program evaluation (PE) units in state 
agencies for vocational rehabilitation (VR) . The STNA strategy is similar, 
however, to training needs identification in any human services area and 
can be easily modified to assess needs in service, administrative, super- 
visory or clerical units within the rehabilitation network. 

Three assumptions are made with respect to the ensuing presentation: 

1. An outside consultant is to be used to perform the assessment. 

2. The consultant used is thoroughly knowledgeable of both the 
responsibility and daily operation of the program evaluation 
unit and with a VR agency. 

3. The consultant is knowledgeable of relevant training programs 
that are available. 

STNA STRATEGY 

The consultee-centered administrative consultation model as described 
by Gaplan (1964) is the basic approach emphasized by most consultants 
undertaking training needs assessments. The primary goal of this system is 
consultee education or enhancement with regard to general program manage- 
ment or development. In the purest sense the goal is to assist personnel 
In improving problem solving skills. The STNA approach differs from this 
model in that the goals are to increase staff capacity for performance of 
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existing and new functions through the acquisition of specific skills and 
knowledge. In the STNA approach the consultant functions both as a catalyst 
to help the consultee determine particular needs and interests (Schein, 
1969) and as an expert to provide training information concerning options 
and providers. 

The STNA approach includes three generic procedural steps: 

1. determine the skills necessary to complete existing staff 
functions and emergency tasks; 

2. assess the present skill level of the existing staff with 
regard to current and proposed functions; and 

3. develop individual training plans to meet the identified 
training needs, 

The approach fits well into Jackson and MacKinney's (1969) characteri- 
zation of an empirical assessment in which a systematic attempt is made to 
collect data on which training needs can be made. 

METHOD 

As this particular approach was developed with program evaluation 
units, all tasks identified in this section are worded specifically for 
program evaluation. The particular methods employed in conducting the 
STNA are detailed below. 

Determine the skills necessary to complete existing staff functions 
functions and new tasks. A six-step process is used to arrive at a concise 
list of skill areas that are required to accomplish the mission of the 
program evaluation unit. 

1. Review the RSA manual chapter, "State Agencies, Studies and 
Evaluation," for suggestions, guidelines and mandates for 
Central Office for PE functioning. 

2. Review the State Rehabilitation Services Operating Procedures 
Manual for suggestions, guidelines and mandates for PE unit 
functioning. 

3. Review the Request for Proposals (RFP) for the Model PE Unit 
Contract to determine its specific requirements. 

4. Review the State Rehabilitation Services Technical Proposal 
in response to the RFP to determine tasks listed beyond 
contract requirments. (Steps 3 and 4 apply only to states that 
have Model Evaluation Unit contracts.) 



5. Have each member of the project staff enumerate the tasks he/she 
is assigned in the performance of his/her normal job duties by 
completing the Task Analysis "Form (Figure 1) . 

6. Meet with the project staff as a group to discover any addition- 
al tasks that may have been overlooked in the previous steps. 

During this phase of the assessment, the consultant engages in a 
familiarizing process in which he/she reviews the directives and guidelines 
related to program mission and complete system mapping activities. System 
mapping (Brown & Wedel, 1974) refers to the development of an understand- 
ing of the area being assessed and the setting in which the activity takes 
place. The STNA strategy assumes that the consultant comes with this 
knowledge or has the ability to quickly develop it by reviewing basic 
program documentation and the individual task analysis supplied by evalua- 
tion unit members. Through this process the consultant is able to insure 
that individual, unit and organizational goals are synchronous thus 
allowing for the opportunity for training to have its maximum impact. 

The final step of involving the entire group in an information dis- 
cussion of tasks is particularly important. This not only validates the 
consultant's impressions, but also serves as a social contact that allows 
the consultant from outside the agency the opportunity to get to know the 
staff personally, establish credibility and promote open communication. 
The group meeting also provides the opportunity to explore the staff 
expectations for training and detail positive training experiences. 

Once the tasks are collected and tentatively validated, they are 
translated into the specific skills that are necessary to accomplish the 
tasks. This part of the process is the most elusive. The translation of 
task statements into competency areas requires the consultant to delineate 
and examine the structural components of each task, determine skill re- 
quirements j and regroup the structural components along the skill 
dimensions . 

In the case of the assessment of a program evaluation unit, the 
skill area grouped for clarity, might include (1) evaluation methods 
(statistics, research/evaluation designs, data processing, measurement); 
(2) management (project management, program planning, budgeting, informa- 
tion systems); and (3) dissemination and utilization (technical writing, 
training, consultation) . 

Assess the skill level of the existing staff. The following five- 
step process is used to examine the skill level of the staff. 

1. Review the curriculum vitae of each staff member identifying 
course work relevant to the identified skills. 



FIGURE 1 



Task Analysis Program 
Evaluation Function 



1. With which particular program evaluation tasks are you directly 
involved? 
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by circling tha appropriate number 1 . 
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3. Tf you have identified a need in one or more of the above areas, 
prior to this evaluation, what specific training options have you 
considered? 



List and task related training you may have received to date. 
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2. Review the work experience and job description of each staff 
member to determine applicability to the identified skills. 
(Permission should be obtained from the individual employee.) 

3. Meet with the staff as a group to ascertain the group's com- 
petence level. This may also be accomplished as a result of 
the group meeting to complete the first objective. (This is 
one of the most difficult tasks, as it requires a judgment 
about the complementary nature of skills and styles of the 
staff.) 

4. Individually interview each staff member to: 

a. determine interest in performing specific tasks; 

b. clarify work and curriculum vitae as well as any present 
training related to specific tasks; 

c. determine competence to perform specific tasks; 

d. identify constraints of time and geographic location that 
may effect the training plan; and 

e. identify any future educational or training goals that 
the staff member may have, 

5. Examine the skill areas of each staff member consistent with 
his/her interests, job assignment and indicated training needs. 

The above employ several different methods of data collection, thus 
meeting Nadler's (1977) admonition of excessive reliance on one method. 
The individual interview does assume major Importance for completion of 
this objective since relationship building is a significant activity at 
this point. Skills similar to those used in initial interviewing are 
required and the atmosphere of professional regard and respect must be 
maintained. In part, this is accomplished by emphasizing the product as 
being the development of new skills and knowledge to increase effective- 
ness and capacity. In other words, the consultant needs to emphasize 
his/her role as a facilitator of the skill building process. 

Of the several very difficult activities which are part of this 
objective, the determination of ability to perform the specified tasks is 
possibly the most challenging. Although the consultant must be a process 
expert, it is also essential that he/she be a content expert in order to 
accomplish this activity. Tasks which routinely confront the staff member, 
day-to-day operations, and examples of their problem solving or samples of 
work must be studied to best determine current skill level. The matching 
of current skill level with required new skill level and the resulting 
discrepancies determine training needs. The consultant must understand 
the activities and tasks well enought to (a) know what skills are required; 
(b) be able to determine how sophisticated the staff member is in a re- 
quired skill; and (c) know the next level of skill attainment appropriate 
to the required work task. Other issues concerning the selection and use 
of consultants have been enumerated by Lippitt (1972) and may be of inter- 
est to the reader. 
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Develop individual training Plans. Before training plans can be pro- 
posed, sources of needed training must be identified. The following seven 
steps are offered as guidelines: 

1. examine catalogues of universities and colleges in the area 
where staff members are located for relevant courses; 

2. review Civil Service course offerings; 

3. review private vendor course offerings or resource consultants; 

4. review the course or workshop offerings of the regional re- 
habilitation continuing education programs; 

5. review professional organization seminar and workshop offerings; 

6. review training programs offered by Rehabilitation Research and 
Training Centers; and 

7. survey major universities with rehabilitation counseling programs, 
for workshop offerings or resource consultants. 

Once this information is organized, training plans are prepared which 
include recommended skill areas for training, alternative plans for receiv- 
ing training, time frames, geographic and travel conditions (usually based 
on agency constraints) , and cost information. (See Figure 2 for sample 
form.) The time frames are usually modified by the staff person's pro- 
fessional time constraints, frequently making short-term training more 
feasible than long term. These recommendations are rank-ordered from 
most important and accessible to least Important and accessible. It is 
usually helpful to also organize the training recommendations or a grid 
or chart (Figure 3) so that patterns of training needed by several people 
or the entire group can be readily observed. 

While this approach to training needs assessment is accomplished 
after the new tasks and program directives are established as a way to 
maximize program success, it can just as easily be used as a program 
planning instrument. Needs assessment as a program planning approach or 
as a diagnostic tool to identify new areas for training not currently 
being provided are equally important thrusts and may provide direction 
for future program development activities, 
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CONCLUSIONS 

The STNA approach is a reliable way of determining the training needs 
of a program evaluation unit within a state agency for vocational rehabi- 
litation. The approach has been used by the authors in two such units. 
Though other approaches may also be viable, the selection of a particular 
approach should not overshadow the need for conducting such data based 
training needs assessments. 

The assessment of training needs should not be considered a one time 
project that merely lists recommendations for training. As Brown and 
Wedel (1974) indicated, "Ideally it is a process of information gathering 
to diagnose changing requirements of the organization and the people in 
it and to develop creative responses to the needs as they are discovered." 
As people, tasks, goals and priorities change, so must the training needs 
and strategies of the organization change. While the initial process may 
not be reinitiated, periodic updates to individual training plans must be 
made. Toward that end the organization needs to recognize and make 
available an opportunity for individual workers, supervisors and staff 
development officers to have input into the identification of training 
needs . 
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